Introduction: Training Deep curriculum learning is a kind of smart agent training in which, first the simple acts, and then, the difficult acts are trained to smart agent. In this study, we proposed a new framework for training Deep curriculum learning to defense-based game in particular Dragon Cave. Materials and Methods: Deep reinforcement learning approach with curriculum learning was used to train an intelligent agent in the game Dragon Cave. curriculum learning paradigm started from simple tasks, and then gradually tried harder ones. Using Proximal Policy Optimization, the intelligent agents were trained in various environments, once in a curriculum-learning environment, and once in an environment without curriculum learning. Then, they started the game in the same environment. Results: The improvement of the agent was observed with Deep curriculum reinforcement learning. Conclusion: It seems that the Deep curriculum reinforcement learning increases the rate and the quality of intelligent agent training in complex environment of strategic games.

Distributed Denial of Service (DDoS) attacks are among the primary concerns in internet security today. Machine learning can be exploited to detect such attacks. In this paper, a multi-layer perceptron model is proposed and implemented using Deep machine learning to distinguish between malicious and normal traffic based on their behavioral patterns. The proposed model is trained and tested using the CICDDoS2019 dataset. To remove irrelevant and redundant data from the dataset and increase learning accuracy, feature selection is used to select and extract the most effective features that allow us to detect these attacks. Moreover, we use the grid search algorithm to acquire optimum values of the model’s hyperparameters among the parameters’ space. In addition, the sensitivity of accuracy of the model to variations of an input parameter is analyzed. Finally, the effectiveness of the presented model is validated in comparison with some state-of-the-art works.

In traditional speech processing, feature extraction and classification were conducted as separate steps. The advent of Deep neural networks has enabled methods that simultaneously model the relationship between acoustic and phonetic characteristics of speech while classifying it directly from the raw waveform. The first convolutional layer in these networks acts as a filter bank. To enhance interpretability and reduce the number of parameters, researchers have explored the use of parametric filters, with the SincNet architecture being a notable advancement. In SincNet's initial convolutional layer, rectangular bandpass filters are learned instead of fully trainable filters. This approach allows for modeling with fewer parameters, thereby improving the network's convergence speed and accuracy. Analyzing the learned filter bank also provides valuable insights into the model's performance. The reduction in parameters, along with increased accuracy and interpretability, has led to the adoption of various parametric filters and Deep architectures across diverse speech processing applications. This paper introduces different types of parametric filters and discusses their integration into various Deep architectures. Additionally, it examines the specific applications in speech processing where these filters have proven effective.

Web application (app) exploration is a crucial part of various analysis and testing techniques. However, the current methods are not able to properly explore the state space of web apps. As a result, techniques must be developed to guide the exploration in order to get acceptable functionality coverage for web apps. reinforcement learning (RL) is a machine learning method in which the best way to do a task is learned through trial and error, with the help of positive or negative rewards, instead of direct supervision. Deep RL is a recent expansion of RL that makes use of neural networks’ learning capabilities. This feature makes Deep RL suitable for exploring the complex state space of web apps. However, current methods provide fundamental RL. In this research, we offer DeepEx, a Deep RL-based exploration strategy for systematically exploring web apps. Empirically evaluated on seven open-source web apps, DeepEx demonstrated a 17% improvement in code coverage and a 16% enhancement in navigational diversity over the stateof-the-art RL-based method. Additionally, it showed a 19% increase in structural diversity. These results confirm the superiority of Deep RL over traditional RL methods in web app exploration.

Web application  rewalls (WAFs) are used for protecting web applications from attacks such as SQL injection, cross-site request forgery, and cross-site scripting. As a result of the growing complexity of web attacks, WAFs need to be tested and updated on a regular basis. There are various tools and techniques to verify the correct performance of a WAF. But most of the techniques are manual or use brute-force attacks, so su er from poor e cacy. In this work, we propose a solution based on reinforcement learning (RL) to discover malicious payloads, which are able to bypass WAFs. We provide an RL framework with an environment compatible with OpenAI gym toolset standards. The environment is employed for training agents to implement WAF circumvention tasks. The agent mutates the syntax of a malicious payload using a set of modi cation operators as actions, without changes to its semantic. Then, upon WAF's reaction to the payload, the environment ascertains a reward for the agent. Eventually, based on these rewards, the agent learns a suitable sequence of mutations for any malicious payload. The payloads, which bypass the WAF determine rules defects, which can be further used in rule tuning for rule-based WAFs. Also, it can enrich the machine learning-based WAFs datasets for retraining. We use Q-learning, Advantage Actor-Critic (A2C), and Proximal Policy Optimization (PPO) algorithms with the Deep neural network. Our solution is successful in evading signature-based and machine learning-based WAFs. While our focus in this work is on SQL injection, the method can be simply extended to use for any string-based injection attacks.

Portfolio management is a challenging task due to the uncertainty and volatility in financial markets, making precise asset allocation and return maximization difficult. This paper presents a novel Deep reinforcement learning (DRL) approach enhanced with fuzzy trend indicators to improve portfolio decision-making. The model was developed using a DRL framework, where a convolutional neural network (CNN)-based policy network learns to optimize asset allocations through interactions with the market. Fuzzy trend indicators are incorporated as additional input features, enabling the model to better capture market uncertainties and ambiguous trends. By providing a more flexible representation of market conditions, fuzzy trend indicators allow the model to dynamically adjust portfolio allocations in response to changing trends, leading to more precise asset allocation decisions and enhanced portfolio performance. The proposed model was trained and evaluated on historical stock data from the Brazilian stock market, covering the period from 2011 to 2020. The dataset includes daily high, low, and closing prices, ensuring a strong foundation for model training and validation. Experimental results show that the fuzzy-enhanced model outperforms some state-of-the-art strategies in terms of both returns and adaptability to volatile market conditions.

In recent years, exponential growth of communication devices in Internet of Things (IoT) has become an emerging technology which facilitates heterogeneous devices to connect with each other in heterogeneous networks. This communication requires different level of Quality-of-Service (QoS) and policies depending on the device type and location. To provide a specific level of QoS, we can utilize emerging new technological concepts in IoT infrastructure, Software-Defined Network (SDN) and, machine learning algorithms. We use Deep reinforcement learning in the process of resource management and allocation in control plane. We present an algorithm that aims to optimize resource allocation. Simulation results show that the proposed algorithm improved network performances in terms of QoS parameters, including delay and throughput compared to Random and Round Robin methods. Compared to similar methods, the performance of the proposed method is also as good as the fuzzy and predictive methods.